Remote direct memory access (RDMA) high performance producer-consumer message processing

ABSTRACT

A method, system and computer program product for remote direct memory access (RDMA) optimized producer-consumer message processing in a messaging hub is provided. The method includes initializing a shared memory region in memory of a host server hosting operation of a messaging hub. The initialization provides for a control portion and one or more data portions, the control portion storing an index to an available one of the data portions. The method also includes transmitting to a message producer an address of the shared memory region and receiving a message in one of the data portions of the shared memory region from the message producer by way of an RDMA write operation on a network interface of the host server. Finally, the method includes retrieving the message from the one of the data portions and processing the message in the messaging hub in response to the receipt of the message.

This application is a Continuation Application of U.S. application Ser.No. 15/265,847, filed Sep. 15, 2016, which claims priority to U.S.application Ser. No. 14/147,511, filed Jan. 4, 2014, which claimspriority to U.S. application Ser. No. 14/144,537, filed on Dec. 30,2013, the both of which are incorporated herein by reference.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to message processing in a message hub andmore particularly to high-speed high volume message processing.

Description of the Related Art

A message hub is middleware disposed within a programmaticinfrastructure that supports the exchange of messages betweendistributed computing systems. In this regard, the message brokeringoperability of a message hub allows application modules to bedistributed over heterogeneous platforms while communicating throughmessage queues managed by a messaging component and reduces thecomplexity of developing applications that span multiple operatingsystems and network protocols. As such, the message hub creates adistributed communications layer that insulates the applicationdeveloper from the details of the various operating system and networkinterfaces.

Unlike messaging appliances that merely consolidate existing messagingservices onto a convenient hardware form factor, the most recentgeneration of messaging hubs are specifically engineered to delivermassive scale communications beyond the enterprise. These messagingappliances deliver publish and subscribe messaging formachine-to-machine communications, communicatively connecting to amultitude of devices and sensors present with the universe of theInternet. These messaging appliances therefore dramatically scale thenumber of concurrently connected devices, enabling large volumes ofevents to be streamed into analytics engines for processing big data.

Of note, most applications utilizing messaging hubs to exchange data,whether between client and server, between processing nodes, processingstages, or threads of execution, require each of low latency—namely oneto one hundred microseconds—high throughput—one hundred thousand to tenmillion messages per second) and also high concurrency—namely multiplepersisted messages in flight at once. However the latency costs, whenusing normal queues in a messaging hub, are in the same order ofmagnitude as the cost of I/O operations to disk. To the extent thatmultiple queues are involved in an end-to-end operation, hundreds ofmicroseconds will have been added to the overall latency of thetransaction. Further, cache misses at the CPU-level, and locks requiringkernel arbitration also can be the result of traditional queuing in amessaging hug and can be very costly.

The message queue approach within a messaging hub demonstratesadditional weakness—namely the potential for a deadlock conditionbetween message producer and message consumer. Specifically, in themessage queue approach, the message producer must not attempt to adddata into the buffer if the buffer is full. Likewise, the messageconsumer must not attempt to remove data from an empty buffer.Consequently, in the message queue approach, the message producer either“sleeps” or discards data when the buffer is full. In the mean time,when the message consumer removes data from the buffer, the messageconsumer notifies the message producer, who in turn can add data intothe buffer once again. In the same way, the message consumer can “sleep”when the message consumer finds the buffer to be empty. When the messageproducer subsequently adds data into the buffer, the message producerwakes the sleeping message consumer. Thus, a deadlock condition canarise where both the message consumer and message producer awaitawakening by the other.

BRIEF SUMMARY OF THE INVENTION

Embodiments of the present invention address deficiencies of the art inrespect to message processing in a messaging hub and provide a novel andnon-obvious method, system and computer program product for remotedirect memory access (RDMA) optimized producer-consumer messageprocessing in a messaging hub. In an embodiment of the invention, amethod for RDMA optimized producer-consumer message processing in amessaging hub includes initializing a shared memory region in memory ofa host server hosting operation of a messaging hub.

The initialization provides for a control portion of the shared memoryregion and one or more data portions of the shared memory region, thecontrol portion storing an index to an available one of the dataportions. The method also includes transmitting to a message producerover a computer communications network an address of the shared memoryregion and receiving a message in one of the data portions of the sharedmemory region from the message producer by way of an RDMA writeoperation on a network interface of the host server. Finally, the methodincludes retrieving the message from the one of the data portions of theshared memory region and processing the received message in themessaging hub in response to the receipt of the message.

In one aspect of the embodiment, the message is retrieved and processedin response to detecting a data available event triggered by the RDMAwrite operation. In another aspect of the embodiment, the data portionsare of equal size and each of the data portions includes a status wordindicating whether or not the data portion is in a valid or invalidstatus indicating whether or not the data has been processed at theserver by the messaging hub. As such, in yet another aspect of theembodiment, the message is not written to the next available dataportion if the status of the next available data portion is valid andthe message is only written to the next available data portion once thestatus of the next available data portion becomes invalid.

In even yet another aspect of the embodiment, the message is received byway of an RDMA write operation on the network interface by firstcomputing from the address of the shared memory region a location of thecontrol portion, second performing an RDMA fetch-add operation on thelocation of the control portion to acquire index data from the controlportion, third computing a modulo of the index data to result in alocation of a next available one of the data portions, and fourthperforming an RDMA write operation of the message to the next availableone of the data portions.

Finally, the messaging hub processes the message by routing the messageto one or more subscribing message consumers. Alternatively, themessaging hub processes the message by discarding the message withoutrouting the message to one or more subscribing message consumers.

In another embodiment of the invention, a data processing system isconfigured for RDMA optimized producer-consumer message processing in amessaging hub. The system includes a host server with memory and atleast one processor and a messaging hub operating in the memory of thehost server. The messaging hub provides communicative couplings througha network interface to different message producing applicationsexecuting in respectively different client computing systems over acomputer communications network. Finally, the system includes an RDMAoptimized message processing module executing in the memory of the hostserver and being coupled to the messaging hub.

The module includes program code enabled during execution to initializea shared region of the memory of the host server to include a controlportion and one or more data portions, the control portion storing anindex to an available one of the data portions, to transmit to themessage producing applications an address of the shared region of thememory, to receive a message in one of the data portions of the sharedregion of memory from one of the message producing applications by wayof an RDMA write operation on the network interface of the host server,and to retrieving the message from the one of the data portions of theshared memory region and direct processing of the received message inthe messaging hub in response to the receipt of the message.

Additional aspects of the invention will be set forth in part in thedescription which follows, and in part will be obvious from thedescription, or may be learned by practice of the invention. The aspectsof the invention will be realized and attained by means of the elementsand combinations particularly pointed out in the appended claims. It isto be understood that both the foregoing general description and thefollowing detailed description are exemplary and explanatory only andare not restrictive of the invention, as claimed.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute partof this specification, illustrate embodiments of the invention andtogether with the description, serve to explain the principles of theinvention. The embodiments illustrated herein are presently preferred,it being understood, however, that the invention is not limited to theprecise arrangements and instrumentalities shown, wherein:

FIG. 1 is a pictorial illustration of a process for RDMA optimizedproducer-consumer message processing in a messaging hub;

FIG. 2 is a schematic illustration of a data processing systemconfigured for RDMA optimized producer-consumer message processing in amessaging hub; and,

FIG. 3 is a flow chart illustrating a process for RDMA optimizedproducer-consumer message processing in a messaging hub.

DETAILED DESCRIPTION OF THE INVENTION

Embodiments of the invention provide for RDMA optimizedproducer-consumer message processing in a messaging hub. In accordancewith an embodiment of the invention, a messaging hub executing in memoryof a server initializes a portion of memory of the server to include adata section and a control section, the control section indexing a freeportion of the data section. Thereafter, a remotely disposed messageproducer seeking to pass a message to the messaging hub can access thecontrol section utilizing RDMA at a network interface of the server andcan write data to the free portion specified by the control section alsoutilizing RDMA at the network interface of the server. The messaging hubin turn can react to the writing of the data to the free portion ofmemory by reading the data and publishing the data to subscribed messageconsumers, all without requiring the use of a messaging queue or theintervention of one or more processors of the server to transfer thedata from the message producer to the messaging hub.

In further illustration, FIG. 1 pictorially shows a process for RDMAoptimized producer-consumer message processing in a messaging hub. Asshown in FIG. 1, a messaging hub 110 can initialize an area of sharedmemory to include a control portion 120 and one or more data portions130. The control portion 120 in particular, can provide an index into anavailable one of the data portions 130 into which a message can beplaced by a message producer 150.

In this regard, a message producer 150 can connect to the messaging hub110 through a network interface 160 of the messaging hub. Uponconnection to the messaging hub 110, the messaging hub 110 can providean address of the shared memory from which the message producer 150 candeduce the control portion 120. Thereafter, the message producer canacquire an index into the shared memory to a data portion 130 byaccessing the content of the control portion 120 utilizing an RDMArequest at the network interface 160.

Once the message producer 150 acquires the index, the message producer150 can write a message 140 to the indexed one of the data portions 130again utilizing an RDMA operation at the network interface 160.Thereafter, the writing of the message 140 to the indexed one of thedata portions 130 can trigger an event 180 in the messaging hub 110sufficient to alert the messaging hub to read the indexed one of thedata portions 130 and to acquire the message 140. Upon acquiring themessage 140, the messaging hub 110 can process the message 140,including transmitting the message 140 to different subscribing messageconsumers 170.

The process illustrated in connection with FIG. 1 can be implementedwithin a data processing system. In more particular illustration, FIG. 2schematically shows a data processing system configured for RDMAoptimized producer-consumer message processing in a messaging hub. Thesystem can include a host server 210 with memory and at least oneprocessor hosting the operation of a messaging hub 220. The messaginghub 220 can be configured to process messages 280 through a networkinterface 230 of the host server 210 according to a publish andsubscribe paradigm from different applications 250 disposed amongstclient computing systems 240 over a computer communications network 270.

An RDMA optimized message processing module 300 can execute in thememory of the host server 210 and can interoperate with messagingmodules 260 coupled to each of the applications 250 in association withthe client computing systems 240. The RDMA optimized message processingmodule 300 can include program code that when executed in memory of thehost server 210, can initialize a region of the memory of the hostserver 210 for sharing by the different applications 250 exchangingmessages 280 with one another. The region can be divided into dataportions of equal size and also the region can include a single controlportion.

The control portion can store an index of a next free one of the dataportions whilst each data portion in addition to storing data in theform of a message 280, can store a status word indicating whether or notthe stored data is valid or invalid—valid meaning that the data is readyto be processed by the messaging hub 220 and invalid meaning that thedata already has been processed by the messaging hub 220. The programcode of the RDMA optimized message processing module 300 also can beenabled during execution in the memory of the host server 210 to respondto a connection by one of the message modules 260 to provide to themessage module 260 an address of the shared memory region, the size ofeach data portion in the shared memory region and a number of total dataportions available for utilization by the message modules 260.

Of note, each message module 260 as a cooperative annex to the RDMAoptimized message processing module 300, can include program code thatwhen executed in memory of a corresponding one of the client computingsystems 240, is enabled to determine from the information provided bythe RDMA optimized message processing module 300 during connection alocation of the control portion of the shared memory region, for examplea location at the beginning of the memory region, and also an address ofthe first data portion in the shared memory region. Thereafter, theprogram code of the message module 260 can acquire an index into a firstavailable one of the data portions.

In particular, the program code of the message module 260 first canperform an RDMA fetch-and-add operation of the network interface 230with respect to the shared memory region location containing the indexof the next available one of the data portions. Subsequently the programcode of the message module can compute a modulo of the total dataportions in the shared memory region to the value returned by thefetch-and-add operation. The result is an index into a first availableone of the data sections of the shared region of memory of the hostserver 210. Optionally, the program code of the message module 260 candetermine if the data portion referenced by the index is marked valid orinvalid so as to ensure that a message is written to the data sectiononly if the index is marked invalid.

Once the message module 260 has identified a data portion that is knownto be in an invalid state, the message module 260 can write the message280 to the indexed data portion utilizing an RDMA write operationthrough the network interface 230. As a result, the state of the dataportion automatically becomes valid. Also, the RDMA write operationtriggers a data available event in the host server 210 that includes anindex into the data portion storing the newly written message 280. Inresponse to the data available event, one or more threads of executionof the program code of the RDMA optimized message processing module 300programmed to react to the data available event can process the message280 by reading the message 280 into memory, marking the data portioninvalid and acting upon the message 280 by transmitting the same to oneor more subscribing ones of the applications 250, discarding the message280, persisting the message 280 or some other operation.

In even more particular illustration of the operation of the RDMAoptimized message processing module 300 and the message module 260, FIG.3 is a flow chart illustrating a process for RDMA optimizedproducer-consumer message processing in a messaging hub. Beginning inblock 305 in reference to the operation of the RDMA optimized messagemodule in the messaging hub, a shared memory region of the host servercan be initialized to include both a control portion and one or moredata portions of equal size, each data portion also including a wordindicating whether or not data is valid or invalid as contained therein.In block 310, an index can be set within the control portion to a firstavailable one of the data portions.

In block 315, a client connection can be received from a message modulein a client computing system acting as a message producer. In response,in block 320 the address in memory of the shared memory region alongwith a number of data portions and a size of each data portion can beprovided to the message module. Thereafter, one or more threads ofexecution of the RDMA optimized message module can await detection of adata available event. In decision block 325, if a data available eventis detected, in block 330 a location of the data in one of the dataportions of the shared memory region can be determined from the dataavailable event and the data can be retrieved and processed as a messagein block 335.

Referring now to the operation of a message module in the messageproducing client, in block 340 a connection can be established with thehost server of the messaging hub so as to produce messages thereto andoptionally consume messages therefrom. In block 345, an address inmemory of the shared memory region of the host server along with anumber of data portions and a size of each data portion can be receivedfrom the RDMA optimized message processing module. Thereafter, in block350 an RDMA fetch-add operation can be invoked on a network interface tothe host server hosting the operation of the messaging hub so as toacquire index data from which an index can be computed to a nextavailable data portion of the shared memory region in block 355.

In decision block 360, it can be determined whether or not the next dataportion is able to receive newly written data based upon the valid orinvalid status of the data portion. If it is determined that the nextavailable data portion is in a valid state, in block 365 the messagemodule can wait while polling the valid or invalid status of the dataportion until the status becomes invalid. In decision block 360, if itis determined that the status of the next available data portion is in avalid state, in block 370 a message can be written to the next availabledata portion and the status set to valid in block 375. The act ofwriting the message to the next available data portion in block 370 alsowill trigger a data available event for handling in blocks 330 and 335.

As will be appreciated by one skilled in the art, aspects of the presentinvention may be embodied as a system, method or computer programproduct. Accordingly, aspects of the present invention may take the formof an entirely hardware embodiment, an entirely software embodiment(including firmware, resident software, micro-code, etc.) or anembodiment combining software and hardware aspects that may allgenerally be referred to herein as a “circuit,” “module” or “system.”Furthermore, aspects of the present invention may take the form of acomputer program product embodied in one or more computer readablemedium(s) having computer readable program code embodied thereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), an optical fiber,a portable compact disc read-only memory (CD-ROM), an optical storagedevice, a magnetic storage device, or any suitable combination of theforegoing. In the context of this document, a computer readable storagemedium may be any tangible medium that can contain, or store a programfor use by or in connection with an instruction execution system,apparatus, or device.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmittedusing any appropriate medium, including but not limited to wireless,wireline, optical fiber cable, radiofrequency, and the like, or anysuitable combination of the foregoing. Computer program code forcarrying out operations for aspects of the present invention may bewritten in any combination of one or more programming languages,including an object oriented programming language and conventionalprocedural programming languages. The program code may execute entirelyon the user's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider).

Aspects of the present invention have been described above withreference to flowchart illustrations and/or block diagrams of methods,apparatus (systems) and computer program products according toembodiments of the invention. In this regard, the flowchart and blockdiagrams in the Figures illustrate the architecture, functionality, andoperation of possible implementations of systems, methods and computerprogram products according to various embodiments of the presentinvention. For instance, each block in the flowchart or block diagramsmay represent a module, segment, or portion of code, which comprises oneor more executable instructions for implementing the specified logicalfunction(s). It should also be noted that, in some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts, or combinations of special purpose hardware andcomputer instructions.

It also will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks. The computer program instructions may also beloaded onto a computer, other programmable data processing apparatus, orother devices to cause a series of operational steps to be performed onthe computer, other programmable apparatus or other devices to produce acomputer implemented process such that the instructions which execute onthe computer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

Finally, the terminology used herein is for the purpose of describingparticular embodiments only and is not intended to be limiting of theinvention. As used herein, the singular forms “a”, “an” and “the” areintended to include the plural forms as well, unless the context clearlyindicates otherwise. It will be further understood that the terms“comprises” and/or “comprising,” when used in this specification,specify the presence of stated features, integers, steps, operations,elements, and/or components, but do not preclude the presence oraddition of one or more other features, integers, steps, operations,elements, components, and/or groups thereof.

The corresponding structures, materials, acts, and equivalents of allmeans or step plus function elements in the claims below are intended toinclude any structure, material, or act for performing the function incombination with other claimed elements as specifically claimed. Thedescription of the present invention has been presented for purposes ofillustration and description, but is not intended to be exhaustive orlimited to the invention in the form disclosed. Many modifications andvariations will be apparent to those of ordinary skill in the artwithout departing from the scope and spirit of the invention. Theembodiment was chosen and described in order to best explain theprinciples of the invention and the practical application, and to enableothers of ordinary skill in the art to understand the invention forvarious embodiments with various modifications as are suited to theparticular use contemplated.

Having thus described the invention of the present application in detailand by reference to embodiments thereof, it will be apparent thatmodifications and variations are possible without departing from thescope of the invention defined in the appended claims as follows:

We claim:
 1. A data processing system configured for remote directmemory access (RDMA) optimized producer-consumer message processing in amessaging hub, the system comprising: a host server with memory and atleast one processor; a messaging hub operating in the memory of the hostserver, the messaging hub providing communicative couplings through anetwork interface to different message producing applications executingin respectively different client computing systems over a computercommunications network; and, an RDMA optimized message processing moduleexecuting in the memory of the host server and being coupled to themessaging hub, the module comprising program code enabled duringexecution: to initialize a shared region of the memory of the hostserver to include a control portion and one or more data portions, thecontrol portion storing an index to an available one of the dataportions, to transmit to the message producing applications an addressof the shared region of the memory, to receive a message in one of thedata portions of the shared region of memory from one of the messageproducing applications server by first computing from the address of theshared memory region a location of the control portion, secondperforming an RDMA fetch-add operation on the location of the controlportion to acquire index data from the control portion, third computinga modulo of the index data to result in a location of a next availableone of the data portions, and fourth performing an RDMA write operationof the message on the network interface of the host server to the nextavailable one of the data portions, and to retrieving the message fromthe one of the data portions of the shared memory region and directprocessing of the received message in the messaging hub in response tothe receipt of the message.
 2. The system of claim 1, wherein themessage is retrieved and processed in response to detecting a dataavailable event triggered by the RDMA write operation.
 3. The system ofclaim 1, wherein the data portions are of equal size.
 4. The system ofclaim 1, wherein each of the data portions includes a status wordindicating whether or not the data portion is in a valid or invalidstatus.
 5. The system of claim 4, wherein the message is not written tothe next available data portion if the status of the next available dataportion is valid and wherein the message is only written to the nextavailable data portion once the status of the next available dataportion becomes invalid.
 6. The system of claim 1, wherein the messaginghub processes the message by routing the message to one or moresubscribing message consumers.
 7. The system of claim 1, wherein themessaging hub processes the message by discarding the message withoutrouting the message to one or more subscribing message consumers.
 8. Acomputer program product for remote direct memory access (RDMA)optimized producer-consumer message processing in a messaging hub, thecomputer program product comprising: a non-transitory computer readablestorage medium having computer readable program code embodied therewith,the computer readable program code comprising: a non-transitory computerreadable program code for initializing a shared memory region in memoryof a host server hosting operation of a messaging hub, theinitialization providing for a control portion of the shared memoryregion and one or more data portions of the shared memory region, thecontrol portion storing an index to an available one of the dataportions; a non-transitory computer readable program code fortransmitting to a message producer over a computer communicationsnetwork an address of the shared memory region; a non-transitorycomputer readable program code for receiving a message in one of thedata portions of the shared memory region from the message producer byfirst computing from the address of the shared memory region a locationof the control portion, second performing an RDMA fetch-add operation onthe location of the control portion to acquire index data from thecontrol portion, third computing a modulo of the index data to result ina location of a next available one of the data portions, and fourthperforming an RDMA write operation of the message on the networkinterface of the host server to the next available one of the dataportions; and, a non-transitory computer readable program code forretrieving the message from the one of the data portions of the sharedmemory region and processing the received message in the messaging hubin response to the receipt of the message.
 9. The computer programproduct of claim 8, wherein the message is retrieved and processed inresponse to detecting a data available event triggered by the RDMA writeoperation.
 10. The computer program product of claim 8, wherein the dataportions are of equal size.
 11. The computer program product of claim 8,wherein each of the data portions includes a status word indicatingwhether or not the data portion is in a valid or invalid status.
 12. Thecomputer program product of claim 11, wherein the message is not writtento the next available data portion if the status of the next availabledata portion is valid and wherein the message is only written to thenext available data portion once the status of the next available dataportion becomes invalid.
 13. The computer program product of claim 8,wherein the messaging hub processes the message by routing the messageto one or more subscribing message consumers.
 14. The computer programproduct of claim 8, wherein the messaging hub processes the message bydiscarding the message without routing the message to one or moresubscribing message consumers.